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ABSTRACT 

The  dependence  of  the  asymptotic  rate  of 
convergence  on  the  order  of  correcting  linear  equations 
in  a  successive  Iteration  scheme  is  studied.   Orderlngs 
which  have  the  same  asymptotic  rate  of  convergence  as 

the  order  correcting  first  equation  first,  second 

th 
equation  second,  ...,  n    equation  last,  are  found  for 

general  linear  systems  and  linear  systems  which  approxi- 
mate linear  elliptic  differential  equations. 
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NYO-7972 

ORDERING   PROPERTIES  OP 

LINEAR  SUCCESSIVE   ITERATION   SCHEMES 

Jack  Heller 

1»   Introduction 

As  digital  computers  become  equipped  with  larger 
and  larger  internal  memories,  the  problem  of  solving 
linear  systems  Internally,  i»e«  without  the  use  of  serial 
input  of  data,  becomes  feasible.   With  this  in  mind,  a 
series  of  experiments  on  the  NYU  Unlvac  were  undertaken 
by  Held  and  Heller  [l]  for  the  solution  of  the  5  and  13 
point  elliptic  difference  schemes.   It  was  shown  in 
these  experiments  that  In  the  second  order  line  scheme 
for  the  5  point  operator,  the  order  of  correcting  the 
lines  in  a  successive  scheme  does  not  affect  the  eigen- 
values of  the  error  matrix.   For  the  13  point  fourth 
order  elliptic  difference  equation  however  it  was  shown 
that  the  order  in  which  one  corrects  the  lines  is  critical. 

In  this  paper  we  will  develop  a  general  theory  of 
ordering  for  successive  iteration  schemes o   Since  the 
matrix  elements  of  the  linear  systems  considered  here 
are  arbitrary,  all  the  ordering  properties  fovmd  will 
apply  to  general  regions,  general  linear  extrapolation 
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(over- relaxation),  and  general  boundary  conditions.   It 
will  be  found  that  the  independence  of  the  eigenvalues 
of  the  error  matrix  for  the  5  or  9  point  Laplace  differ- 
ence scheme  does  not  depend  on  making  special  assumptions 
about  the  region  of  solution  or  boundary  conditionso   In 
the  case  of  the  general  n   order  linear  system  we  will 
find  that  any  cyclic  permutation  of  the  order  of  succes- 
sive iteration  in  the  solution  of  the  equations  will  give 
rise  to  error  matrices  all  having  the  same  eigenvalues. 
Finally,  in  the  case  of  the  fourth  order  successive  line 
schemes,  the  number  of  orderings  giving  rise  to  the  same 
eigenvalues  for  the  corresponding  error  matrices  will  be 
enumerated.   This  procedure  of  enumeration  can  be  applied 
to  any  linear  successive  iteration  scheme© 
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2«.   The  Error  Matrix  for  a  Linear  Successive  Iteration 
Scheme o 

Given  a  system  of  linear  equations 

(1 )  AX  =  B 

to  be  solved  by  a  successive  Iteration  scheme,  we  split 
the  matrix  A  Into  a  sum 

(2)  A  =  R  +  3 

where   R   is  in  some  sense  a  lower  triangular  matrixo 
This  split  suggests  the  Iteration  scheme 


(3)     RX^^-'l^  =  B  ~  SX^^^ 


where   X     Is  the   v    iterate,  having  started  with 
an  initial  guess  X   o   In  recent  years,  techniques 
for  the  solution  of  linear  systems  by  the  successive 
scheme  fonnally  given  by  (3)  have  received  great  atten- 
tion [2,3,U]* 

Defining  the  error  after  the   v*'^   Iterate  as 


(1^)     e^^^  =X  ^  X^^^ 


we  obtain  from  (3)j  (2)  and  (1); 


(5)  e^^-^l^  =  -R-^  Se^^^ 

The  matrix   -R""''S   is  called  the  error  matrix,  and  its 
eigenvalues  satisfy  the  determinental  equation 

(6)  det  IXR  +  S|  =  0   • 

Implied  In  the  splitting  of  the  matrix  A  of  (1 ) 
is  a  given  order  of  correcting  the  elements  of  the  vector 
^(v+1)^   Since  R  is  lower  triangular,  the  first  component 
of  X   "^   is  computed  first.   Then  the  second  component 
of  X       is  computed  using  the  corrected  first  component, 
This  procediore  is  continued  imtil  all  the  components  of 
X  "^      are  computed.   If  we  choose  another  ordering  in 
which  x^   is  not  corrected  first,   Xp  not  second,  etc, 
and  new  values  are  used  whenever  available,  we  will  still 
have  a  successive  iteration  scheme  but  the  error  matrices 
will  differ.   In  general,  therefore,  we  would  not  expect 
the  eigenvalues  to  be  the  same.   We  will  refer  to  the 
scheme  given  by  (3)  as  the  "usual"  scheme.   All  others 
will  simply  be  referred  to  as  "ordered"  schemes. 

Let  A  =  (a..).   Then  from  (6)  we  have 

^12      • • •        ^In 


( 7  )  ?iR+S  =  jV-  =  • 


^^22     • • •        ^2n 


^^n-l,n-l   ^n-l,n 


n,l  n,n-l      nn 
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st 
If  we  have  another  ordering,  say  the   i,    equation 

corrected  first,  the   ip    equation  second,  etc.,  then 

s  t  s  t 

at  the   v+i    iteration,  the  i-,    equation  is 

(8)  a^  ,   xf-^)  =-r-     a.  .  x(;)  .  b^ 

the   ip    equation  is 

(9)  a^.  ,    x(^+l)  +  a,  .  xf^+l)  =  -  > a.  .x(^)  +  b. 

^2^2  ^2        ^1^2  ^1  jTVig   ^2J  J       ^2 

and  in  general 

(10)  ^ZIZ         a^.    .      x|^+l)  =  -         ZZZ  a.     .x(/)4-b. 

n=l  p^n     ^s  n,j=l  ^n"^    "^  ^n 

p,s=l,2,  .  .  .  ,n  jVi    ,i         ,...,i 
p=s 

If   we  define 

1;  1^  -  j  >  0 

(11)  ^(i^,j)    = 

0;    i^   -    J    <   0 

Then  the   error  matrix        6"  R+S     becomes 


^^11 

(12)      OT+S   =      I  [t?(l-e(j,i^))  +  ^(j,i^)/aj. 

[^(l-4(l,,J))  +  e(l,,J))a.^. 


J^n 


8   - 


Prom  (11)  and  (12)  we  see  that  If  an  element  a.  . 
Is  multiplied  by  cT   its  transpose   a.^   is  not.   Setting 


l-'C 


(13) 


o-(l-^(i^,J)  +  ^(1^,0))  =  cr 


■in»J' 


where 


(li+) 


tr(i„,j)  = 


n 


i;  VJ  >  0 


Oi  i^-j  <  0 


we  have 

Theorem  I«   In  a  successive  iteration  scheme  for  the 
solution  of  a  linear  system,  if  the  order  of  correcting 
the  equations  is   i,,ip,..e,i  ,   the  error  matrix  has 


o   a 


12 


•   •   « 


<f       a 


In 


"^23 
^^22    '^       ^23 


^^2n, 
cr       a 


2n 


{i^)Z 


^"^,n-: 


n-l,n 


•  •  • 


(fa. 


h-l,n-l 


o^    'a 


■n-l,n 


ycr^"^i^a 


nl 


l-'C 

cr   ^-l*^a    ,  o-a 

n,n-l   nn 


We  look  upon  the  error  matrices  (7)  and  (l5)  as  Lambda- 
matriceso   Then  the  maximum  absolute  value  (spectral 
radius)  of  7*.   and  C      obtained  by  setting  the 
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determinant  of  these  matrices  equal  to  zero,  determines 

the  rate  of  convergence  of  the  Iteration  scheme  .   If 

K|     =  Icrl    ,   the  rates  of  convergence  of  any  ordered 
'  'max      'max' 

scheme  and  the  usual  scheme  will  be  the  same.   Two 
questions  present  themselves: 

1»   Under  what  conditions  does   Ul^^^  =  Ic^lrnax*^ 

2.   To  determine  mln  (l^^  l^ax' '  "^'max  ^  *  l^'max  ^  '"^'max" 
Question  1  is  comparatively  simple,  since  we  need  make  no 

ass-umptions  aoout  the  elements   a^  .   of  the  linear  system* 
On  the  other  hand,  to  answer  question  2,  we  need  some 
knowledge  of  the   a.  ..   We  will  be  content  here  to  answer 
the  first  question. 

Let  us  consider  the  following  transformation  of  an 

error  matrix  y        . 


5ZS-- 


"a^^as              V^ 

J 

\ 

\ 

-f..,J 

L. 


Cf 


■'X. 


■'•o-ij>-i)/ 
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o-'^-^a 


31  ~^   ^  ~- 

era. 


.^''"^V..  c5-a 


where 
(17) 


r,j  =  v^.  - g  r,. 


1-1 

"^  k+1 


Setting  the  determinants  >    and  S^ S"   equal 

to  zero  will  give  rise  to  identical  values  of  <T  >      We 
see,  therefore,  that  if   P^  .  =  0,   the  error  matrix  of 
an  ordered  scheme  (l5)  and  the  usual  scheme  (7)  will  have 
the  same  eigenvalues.   In  the  case   P.  .  f-   0,   the  eigen- 
values of  >    and  S^~~  S~   would  be  equal  only  for 
special  values  of  a. .,   e.g.   a, .  =  0,   Therefore  for 
general,  a.  .,  we  have 

Theorem  IIo   For  any  successive  linear  iteration  scheme 
with  arbitrary  a.  .   the  eigenvalues  of  the  error  matrix 
will  be  the  same  as  the  eigenvalues  of  the  corresponding 
usual  scheme,  if  and  only  if 

(18)     t^.  .  -  iZ   k  1.-+1  =  °       i  <  J  =  2,3,i4.,..o,n 

k=l   .^  j-t"- '  -1- 

for 


(19)     -C^.   =0  or  1. 
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th 
3o   Orderings  of  General  n   Order  Systeins  having  the 

Same  Eigenvalues  as  the  Usual  Ordering . 


To  find  those  orderlngs  whose  error  matrix  has  the 
same  eigenvalues  as  that  of  the  usual  ordering,  we  need 
only  solve  (l8)  with  condition  (19)  and  find  which 
orderlngs  these  solutions  determine.   The  system  (l8)  is 
a  set  of  equations  whose  coefficients  are  integers  and 
whose  solutions  are  integers «,   Linear  systems  of  this 
type  arise  in  the  study  of  linear  graph  theory  and 
combinatorial  topology  [5^*6] «   The  fact  that  this  type 
of  equation  arises  here  is  not  surprising*   The  form  of 
the  error  matrix  for  a  general  ordering  circumvents  the 
problem  of  keeping  track  of  corrected  and  uncorrected 
values,  which  in  terms  of  fitting  the  corrected  and 
uncorrected  values  into  a  given  mesh,  can  be  viewed  as 
a  problem  in  combinatorial  topology.   We  will  not  malce 
any  use  of  the  general  theory,  but  will  simply  solve 
the  special  equations (l8)    » 

Prom  (l5)  we  see  that  the  niomber  of  unknowns   f.  . 
counting  along  diagonals  is 

(20)    N^  =  (n-1)  +  (n-2)  +  .,,  +  2  +  1  =  2l|zi) 

Prom  (16)  we  see  that  the  number  of  equations  is 

1«   By  a  solution  of  (l8)  we  will  mean  a  solution  for 
which  all   'C'j.=Oorl« 
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(21)    N^  =  (n-2)  +  (n-3)  +  ...  +  2  +  1  =  (^-^H^-^) 


Hence 


(22)    N^  -  Ng  =  n  -  1  . 

Therefore,  since  the  equations  are  homogeneous, 
there  are  always  non-trivial  solutions  of  (l8)  and  we 
need  only  find  those  for  which   t".  .  =  0  or  1.   We  also 
note  that  there  can  be  at  most  one  none  zero  term  in  the  sum 

Let  us  consider  the  equations  for   i  =  1. 
^13  -  '  ^12  *  ^23'  =  ° 


^lU  -  '  ^12  -^  •^23  ^  ^31^'  =  ° 


(23) 


In  the  equation  for   j  =  n,   the  term   (t,  p+...  +  'C_-|  ) 
must  be  either  0  or  1.   If  it  is   0,   all  the  other   'cr'.  . 
terms  are  zero  because  each  bracket  in  the  proceeding 
equations  of  (23)  is   0.   Also  all    "C    are  zero  for 
all  terms   (  '^j^^^^.^  "*"  ^l+li+2  '*'••'    "^    "^n-ln^  ^^'^   hence 


all   "C.  .  =  0.   The  ordering  corresponding  to  this  solution 
is  the  usual  ordering. 

If  the  sum  term  in  (23)  for  the  equation   j  =  n  is 
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not   0,   one  of  the   ^••+-1   terms  must  be   1,   say 
T^  ^  . ^  =  1»   Then  the  remaining  equations  In  (23)  are 
determined,  for  In  each  equation  of  (23)  the  sum  term 
Is   0   or  1   and  so  the  first  term  of  the  equation  Is 
either  0   or  !«   The  same  is  true  for   1=2,   since 
all  the  sum  terms  are  determined  and  the  remaining  term 
In  the  equation  Is  then  determlnedo   This  same  argument 
then  applies  to   1  =  3fh->'»»>'^»      ^^  thus  see  that  when 
one  of  the  terms  In  the  sum  term  of  (23)  for   1=1   Is 
chosen  as   1,   all  other  tr .  .      are  determined  and  are 
different  for  each  choice  of  ^i?ff+-]»      Since  we  can 
choose  the  '^j  0+-\      term  In  the  sum  term  of  (23)  for 
1=1   In  n-1  ways,  we  have  n  different  solutions 
counting  the  trivial  solution   T . .  =  Oo   We  thus  have 
the 
Lemma?   There  are  but   n   solutions  of 


r^j.  -  |Z  -t-kk+l  =0        i  <  J  =  2,3,..o,n 


for  which 

irl  .  =  0  or  1, 

To  see  which  orderlngs  correspond  to  these  solutions, 
consider  ^^jt+i  ~  -^^  Then  we  have,  by  observing  the  form 
of  (23)  for  all   1, 


-  Ik   - 


^J  =  ° 

j  <  £ 

=  1 

J  >  / 

^20-   =  ° 

J  <-^' 

=      1 

J>/ 

2  <X 


r. J.  =  0       j  </  1  <^ 

=  1       J  >/ 


and  if  1  > /.  r.  .  =  0. 

The  error  matrix  (l5)  for  this  solution  is 


<^^m 


era,,     a,i.      -  -  -     a,^.,    ^^^  ^O.,^,,     -    -    - 


^^"^    '^A-X      ^A-i      ^^^i  <^0--^ 
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The  equations  that  can  be  corrected  first  for  some 
ordering  are  those  which  have  (f      on  the  diagonal  term 
of  the  error  matrix  onlyj  the  next  equations  that  can  be 
corrected  are  those  which  have  a  cT   on  the  diagonal 
term  of  the  error  matrix  and  a  (T   in  the  column   1-^ 
equal  to  the  first  equation  corrected}  ooo,   for  the  <f 
terms  appear  with  the  corrected  values*   We  thus  see 
from  (24)  that  the  order  of  correcting  the  equations 
corresponding  to  one  of  the  solutions  of  (I8)  Is 

/Cj    ^"^i         /^"*"2  J   oeO|i   n,   1,   2,   ••a;     A--le 

Since  this  is  true  for  any  /- ^      we  have 

Theorem  III»   Only  cyclic  permutations  of  the  ordering  of 
a  linear  system  solved  by  a  successive  iteration  scheme 
will  give  rise  to  the  same  eigenvalues  of  their  corres- 
ponding error  matrlceso 

This  result  is  not  surprlsinge   If  we  started  with 
the  jt    equation  and  corrected  the   /,£+!,.  o.,n 
equations  and  then  started  the  iteration  scheme  as  a 
usual  scheme,  it  would  be  tantamount  to  starting  with  a 
different  initial  guess,  and,  as  is  well  known,  the 
eigenvalues  of  the  error  matrix  are  independent  of  the 
initial  guess.   In  the  case  of  special  linear  systems 
derived  from  differential  equations,  the  orderlngs  are 
not  always  obvious,  and  so  we  will  proceed  to  the  study 
of  these  cases, 
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U.,      Ordering  Properties  of  Linear  Difference  Schemes 
Derived  from  Elliptic  Differential  Equations  Solved  by 
Special  Block  Iteration  Schemeso 

Linear  systems  derived  from  elliptic  differential 
equations  have  a  systematic  form  to  their  matrix}  this 
form  Is  exhibited  by  the  uniformity  of  the  zero  and 
non-zero  elements.   For  example,  the  difference  approxl- 
matlon  to  57(1*  dx^^  ~  f(x)  would  lead  —  in  the  simplest 
approximation  --  to  a  linear  system  of  the  form 


(25) 


f' 


&-,  -1    a.,  Q   (J  •  •  • 


Q.p-1     ^00    ^O'X  u  •  •  • 


a 


32   ^33  ^3h 


0 


The   a ,  .   depend  on  the  mesh  spacing  and  D.   For  higher 
order  schemes  in  one  dimension,  there  will  simply  be  more 
diagonal  rows  without  zeros.   In  two  dimensions  line 
schemes  derived  from  2p   order  elliptic  differential 
equations  give  rise  to  linear  systems  whose  matrix  is  of 
the  form 


-  17  - 


'^11    -^12 


^ip+l   ° 


'21    ^22    ^23   *••  ■^2p+l  ^2p+2   ° 


^p+11  ^p+12 


A 


p+22 


p+lp+1 


%+22p+l    ° 


^+2p^2 


\ri£2p*2. 


U  a  •• 


where  the   A.^   are  matrices  of  the  coefficients  relating 
points  along  the  line  and  A.  .   (i/j)   are  matrices  of  the 
coefficients  relating  points  off  the  line.   The  matrices 
Aj  J   are  square  matrices  with  the  number  of  rows  equal  to 
the  number  of  points  along  the  line. 

As  in  Section  2  we  see  that  the  form  of  the  error 
matrix  for  a  successive  scheme  is 


r, 


^'^^'A,p.,     0     - 


t: 


a-^'*^A,p.,     0 


-    0 


0 


Jiwpti 


p»i  p+' 


o 


o     <^'^''  A 


pn 


(T-A 


nrv 
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By  a  similarity  transformation  similar  to  (l6 ) 
except  with  diagonal  elements  multiplied  by  unit  matrices 
of  the  proper  size,  we  find,  that  a  sufficient  condition 
for  the  eigenvalues  of  the  error  matrix  of  any  ordering 
to  be  equal  to  eigenvalues  of  the  usual  ordering  is 

(27)  r  .  -  ^  1^  =0     1  <  j  =  2,3,...,fp+i'}   . 

We  again  argue  as  before  that  this  condition  is  necessary 
also,  because  the   A.  .   are  general  and  not  zero  and  we 
have  the  co-unterpart  of  theorem  II; 

Theorem  II  »   For  any  successive  linear  iteration  scheme, 
where  the  error  matrix  is  of  the  form  given  by  (26),  the 
eigenvalues  of  the  error  matrix  will  be  the  same  as  the 
eigenvalues  of  the  error  matrix  of  the  corresponding 
usual  scheme,  if  and  only  if 

(28)  r  .  -  ^  V^^^^   =  0     i  <  j  =  2,3,...,  Ul]  ^ 

•J    k=i  L  n>min« 

for 

(29)  r.  .  =  0  or  1. 

The  only  effect  of  having  the  triangles  of  zeros  in 
the  upper  right  and  lower  left  portion  of  the  error  matrix 
is  to  limit  the  number  of  equations  for   '^.  .»   By  counting 
along  the  diagonals  we  see  that  the  number  of  unknown 

'^ij  ^  ^^ 
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(P)  =  r^_n  ^  +  (r.-9\     +  ...  +  fn-T.")  =  nr.  -  P  ^  P"^^  ^ 


(30)    N^P^  =  (n-1)  +  (n-2)  +  ...  +  (n-p)  =  np 


u 


2 


and  the  nimber  of  equations  by  counting  along  the  diagonal 
of  the  similarity  transformed  error  matrix  is 

(31)  nIp^  =  (n-2)  +  (n-3)  +  ...  +  (n-p)  =  np-(n-l)-  £i|±ll 
The  excess  of  iinknowns  over  equations  is 

(32)  N^P^  -  N^P)  =n  -  1   , 

which  is  the  same  as  the  case  of  the  full  matrix.   Because 
(28)  is  a  homogeneous  system  with  more  unknowns  than 
equations,  there  are  always  non-trivial  solutions. 

Whether  there  are  solutions  for   Z^.  ,  =  0  or  1   can 
be  settled  by  straight  forward  exhibition,  and  even  the 
number  of  different  solutions  can  be  obtained  in  the  form 
of  a  generating  function.   However,  the  number  of  solutions 
of  the  v.  .      equations  (28)  is  not  equal  to  the  niimber  of 
orderings,  for  there  is  not  a  one  to  one  correspondence 
between  them.   We  can  easily  construct  a  simple  example 
to  show  this  multi-valued  property  of  the  orderings  as 
a  function  of  the   f.  .   solutions© 

If  we  consider  the  case  of  n  =  5   and  p  =  2,   we 
find  that  the  solutions  of  (28,29)  are 
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r. 


12 


r; 


13 


r; 


23 


TX 


24 


r: 


3i| 


V, 


35 


'1^5 


(a) 

1 

1 

0 

0 

0 

0 

0 

(b) 

1 

1 

0 

1 

1 

1 

0 

(c) 

1 

1 

0 

0 

0 

1 

1 

(d) 

0 

1 

1 

1 

0 

0 

0 

(e) 

0 

1 

1 

1 

0 

1 

1 

(f) 

0 

0 

0 

1 

1 

1 

0 

(g) 

0 

0 

0 

0 

0 

1 

1 

(ii) 

0 

0 

0 

0 

0 

0 

0 

Corresponding  to    these    solutions  we   have   the  following 
error  matrices    (where   we   have    suppressed   the     A.  .     which 
are    irrelevant   in  the   present  context)    and  their  corres- 
ponding  orderings 


(a) 


1^ 
/l 


cf  o  0  0^ 

CT  1  1  0 

0-  cr  1  1 

c  cf  c  1 


1 

0 

\p  0  cT  0-  oV    i^.    5    5 


orderings      (b) 

S     h     3  I 

111 

2  2      2 

3  3     U 


(c) 


(d) 


croo\    5    k    5    U    fa- 1 

110        112     2o-cr 


ord 

eri 

ngs 

'cT  cT  (5-  0    o\ 

5 

5 

1  cTl  cr  0 

3 

2 

1  0-  cT  a-  a' 

h 

k 

0   1   1  C  1  j 

1 

1 

0    0    1    C  c^ 

2 

3 

cr  1  (TO  o\ 
c  a-  cr  cr  0 

h 

2 

2 

3 

3 

5 

5 

i+ 

U 

5 

1    1    0-  1   1 

1 

1 

1 

1 

1 

0  1  0-  cr  1 

2 

3 

3 

2 

2 

0    0    C  0-  (5"/ 

3 

4 

5 

5 

h 

21 


(e) 


(f) 


/cr  1  1  0  o\ 


0-  cf  1  cr  0 
cr  cr  cr  cr  cT 
0  1  1  cr  1 


3      2      2 

5     5     5 
111 


VoOlcrcr/     2     1;      3 


(h) 


/^ 


^  0   cr  cr  cr  cr  I   i? 
Y   0   1   1   0-/     1 


V 


cr  1  1  0  0  \ 

1 

cr  cr  1  1  0 

2 

cr  cr  cr  1  1 

3 

0  cr  cr  cr  1  j 

1+ 

0  0  cr  (f  67     5 


We  see  that  every  solution  of  the   ZT^  .   equations  (28), 
except  the  trivial  solution  which  corresponds  to  the 
usual  ordering,  gives  rise  to  more  than  one  ordering. 
There  are  two  questions  to  be  asked j   (l)  how  many 
different   f.  .   solutions  are  there?  and  (2)  how  many 
orderings  correspond  to  a  given   'C\  .   solution? 

Before  going  on  to  these  general  questions,  there  is 
one  case  of  interest  that  we  can  easily  answer:   the 
case  of   p=l  [10].   This  is  the  case  in  which  the  group 
of  equations  being  corrected  is  connected  to  only  two 
neighboring  groups  of  equations;  e.g.  the  line  schemes 
for  the  five  and  nine  point  Laplace  difference  operator 
in  two  dimensions  and  the  plane  schemes  for  the  seven 
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and  fifteen  point  Laplace  difference  operator  in  three 
dimensions. 

In  the  case  of   p=l   the  mamber  of  unknowns  from 

(30)  is  N    =  n-1   and  the  number  of  equations  from 

(31)  is  N    =0.   Since  there  are  no  restrictive   TlT.  . 

e  ij 

equations,    we   could   choose    the   first    equation  to   be 
corrected   in   any  one   of     n     ways,    the   second   in  any 
one   of     n-1     ways,    ..«    •      We   therefore   have 
Theorem   IV«        If   the    error  matrix  of    a  linear   successive 
iteration    scheme   is   a  triple   diagonal   form 

/  ^12 

/era, ,  cT     A-.  2      0  .  • .  , 

/    ^"xZ  23 

(T  A^-L   tfA22  ^       A23   0    ... 

0  cr         A32  ^A^o        *^  ^3ii  0  ••• 


nn-1        i 


the   number  of  ways   in  which   the   equations   can  be  ordered 
and  give   rise   to   error  matrices  which  have    the   same  eigen- 
values  as   the  usual   ordering  is     nl . 
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5«  The  Number  of  Different   tT^ .   Solutions  for  a  General  p 

Let  us  first  consider  the  case  of  p=2,   which  corres- 
ponds to  equations  coupled  in  groups  of  five,  e.g«  the  line 
schemes  for  the  bi-harmonic  equation  in  two  dimensions* 
The   f,.   equations  are 

(33)    1-13  -  ^^12  -^  ^23^  =° 


(35)  ^ii+2  -  ^^ii+1  ■"  '^1+11+2^  =° 

(36)  r'^.2n  -  (T^n-2n-l  "^   "^n-ln^  =  ^   ^ 

The  equations  are  coupled  via  one   TT^^   from  one  equation 
to  the  next,  namely  the   'Z:'i+ii+2-   Therefore  a  solution 
of  the   i*^  equation  will  have  some  bearing  on  the  solution 
of  the   i+1^    equation. 

Let  us  make  the  following  definitions*   let   ^^  be 
the  number  of  different  solutions  for  the  first   i   equa- 
tions with  the  first  term  i^  the  bracket  of  the   i 
equation  equal  to  one;  let   ']^  have  the  corresponding 
meaning  for  the  second  term  in  the  bracket  J  and  let 
^        be  the  number  of  different  solutions  for  the  first 
i   equations  with  the  bracket  term  of  the   i     equation 
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equal  to  zero »   We  consider  the  first  equation  (33)« 
The  solutions  are 


12 

^13 

^23 

0 

1 

1 

1 

1 

0 

0 

0 

0 

and 

When  we  consider  two  equations  (33>3i4-)j  we  obtain  new 
solutions  If  the  coupled  variable  (In  this  case   '^o'l^ 
Is  zero,  for  then  the  bracket  term  can  either  be  zero 
or  one  giving  two  possible  solutions  for  each  previous 
solution  with    T^-,  =  Oo   The  solutions  are.  In  tabular 
form 


12 

^13 

^23 

^24 

Su 

0 

1 

1 

1 

0 

1 

1 

0 

1 

1 

1 

1 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

0 

0 

For  this  case 
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12  =2 

(^2=2. 

In  general  whenever  there  is  a  one  for  the  coupled  vari- 
able no  new  solutions  sirise  and 

(37)  i   =  lti 

In  the   i^^  equation  the  uncoupled  variable  of  the 
bracket  term  can  be  one  for  every  solution  with  the 
coupled  variable  zero.   The  coupled  variable  is  zero 
only  when  the  bracket  term  of  the  previous  equation  is 
zero  or  the  first  term  of  the  previous  equation  is  one. 
We  therefore  have 

(38)  f?=<5'i-iMtl 

In  the   1^^  equation  the  bracket  terms  can  be  zero  if 
the  coupled  variable  is  zero.   The  coupled  variable  is 
zero  only  when  the  bracket  term  of  the  previous  equation 
is  zero  or  the  first  term  of  the  previous  equation  is 
oneo   We  therefore  have 


(39)        ^1  =  ^1-1  ■"  li-l 


1-1 

th 


The  number  of  different  solutions  for  the  first   1 
equations  is 


1 
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because  the  sum  on  the  right  represents  all  the  possible 
values  of  the  bracket  terms  of  the   ^  .   (p=2) 
equations. 

We  can  find  a  somewhat  simpler  expression  for  the 
number  of  solutions,  if  we  consider 

using  (37),  (38),  (39)  and  (ij.0).   Assuming  we  knew  7i+l» 
we  could  solve  (i|l)  as  a  difference  equation: 

([j.2)  N.^^^    =  const   +  N,^^^    +  ZZ   1?      • 

1  J-  k=2 

(2) 

From   the    above   case   of      1=1,      we    see   that      N,         =  3« 

Hence   the   constant    in   (i;2)  must   vanish   and 

(u)         Np)  =  3+izik  =  2+n'?k 

^  k=2    ^  k=l     ^ 

since      'f^   =  1.      Prom  equations    (37),    (38)    and    (39) 

(hk)  1  ;     =  1  )  I   2'^ 

Using   the   values   for      1=1      and   iterating,    we  have 

l}\  10        1   \"~*       /  1 

(1+5) 

11/  V 1 


For  a  system  of  n  equations,  the  maximum  i=n-2. 

(2  ) 

If  we   call     N  the   ni:imber   of   different   solutions   of 
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the  "ZT.  .   equations  for  a  linear  system  of  n   equations, 
we  have 

(U6)  N^2)   =  2   ^YZll 


k=l      ^ 


and 


(1+7) 


pop 

In  the   case   of     n=5,      we  have      ^^   =  1,      ^p   =  2,      '^o   =  3, 
and   then   from   (l+S),      N^^     -  2+1+2+3  =  8     which  verifies 
the  example   of  Section  [|.o 

We  notice    an    Interesting   sidelight   from   (37),    (38) 
and   (39); 

ikQ)  ll  =  lti  -  1i-2  « 

A   sequence  of  terms  in  which  the   1    term  is  the  sum 
of  the  two  previous  terras  is  called  a  Fibonacci  sequenceo 
We  here  are  interested  in  the  sum  of  a  Fibonacci  sequence. 
Series  of  this  type  are  called  recurring  series j  both 
Fibonacci  sequences  and  recurring  series  have  received  a 
great  deal  of  attention  by  mathematicians  since  their 
discovery  in  the  early  middle  ages  [8]« 

In  a  similar  manner  we  can  find  a  generating  function 
for  the  number  of  solutions  of  the  XT.   .   equations  and  a 
general   p  (<  n-1).   We  define  as  before,  ^.       as  the 
number  of  different  solutions  for  the  first   i  equations 
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,th 


and  all      j     with   the     ji  term   In  the  bracket   of   the 


th 


iquatlon   and   maximum      j      equal    to  one,    and     <^, 


th 


as  the  number  of  different  solutions  for  the  first  1 
equations  and  all  j  with  the  bracket  term  of  the  i 
equation  and  maximum   j   equal  to  zero. 

Only  the  solutions  for  the   i    equation  and 
maximum   j   need  be  considered,  for  as  we  saw  before 
the  other  equations  for  this   i   are  completely  deter- 
mined.   The  coupling  of  the   i    equation  for  maximum 
j   takes  place  as  before  in  the  bracket  term  only.   The 
maximxim  value  of   j   is   i+p+1   (<  n).   Two  successive 
equations  for  maximum   j   are 


^^•^^  ^ii+p-M"(qi-Hl  +  q-.li+2-^--°-^^i+pi+p-M^  =  ° 


(50)  ^i+ii+p+2-^^i+ii+2"^*'"'^'^+pi+p+l'^'^l+p+li+p+2^    °' 


The   coupling   is   of   the   same    form   as    in   the    case      p=2| 
and  we   easily   show  by   similar    arguments    that 


(51) 


^i        1i=l 
'1         'i-1 


V  ir  =  Cx 


and 
(52) 


fl- 


^i-1   ^  ^\-l 
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=  O". 


which  are  the  counterparts  of  (37)  and  (38),  and  (39)» 

The   nimber  of  different   solutions   for   the   first      1 
equations   and  all      j      is 


th 


(53) 


N 


[P)    =     1\^     1I^   ...    +     11   + 


<y. 


because  the   siun  on  the  right   represents    all   the   possible 
values   of   the  bracket   terms   of    the       c     .      equations   for 
inaxlim;m      j.      As  before   we   find  that 


(54) 


N 


(P)    _   ^(P)   =      -/P 
i+1       ^^i  'i 


+1 


and  the   solution,    looked  upon   as    a  difference   equation 
for     N^P^      is 


(55) 


nJp^  =  P  + 


k=l 


7P 


rCp)  = 


which  satisfies  the  condition  N-j^  =  p+1. 

Finally,  as  before,  the  generating  function  portion 
of  the  solution  is 


(56) 


0 


i-l 


0   0 


0 


»   o   • 


For  a   system   of     n      equations    the  maximuin     i=n-p. 
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and   the   total  n-umber  of    solutions   is 

n-T 


(p) 


(57)  N^P'    -  P   +IZ     1?  (p  <  n    -   1) 

where   '/P  is  evaluated  by  using  the  generating  fiinction 
(56). 

We  can  sum  up  the  results  in  the  following 
Theorem  V»    The  number  of  different  solutions  with 
■C^.  .  =  0  or  1   of  the  equations 

(58)  Cij-(q.^i+r^^i  i+z'-'-'-'-^i+P  i+p+i^  =  °    ^^'^*''^*^'*'f n^in. 


is 

(59)  N^P^=p+|f^     fP  2<p<n-l 

k=l        ^ 

where   'j^   is  determined  from  (56)» 

Since  the  n\imber  of   C.  .   solutions  are  a  lower 
bound  to  the  number  of  orderings  for  which  the  eigen- 
values of  the  error  matrix  are  the  same  as  the  usual 
ordering,  as  can  be  seen  from  the  example  in  Section  I|., 
we  have 

Corollary  V«    1*   The  number  of  orderings  ^r,  which 

give  rise  to  the  same  eigenvalues  as  the  usual  ordering 
satisifes 

nl  >  N^P^  >  N^P^      2  <  p  <  n  -  1  . 

Prom  (59),  when  p=n-l,   the  sum  term  reduces  to 
^^~     =1,   and  hence  we  have  the 
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Corollary  Ve  2,   The  number  of  different  solutions  with 

^.  .  =  0  or  1  of  the    ZTl  .   equations  for  the  full 
1 J  -1-  J 

matrix  is 


This,  of  course,  is  the  result  that  we  foxond  in  Section  3» 

The  interesting  sidelight  we  found  for  p=2  has  its 
generalization,  for  we  obtain  from  {$!)    and  (52)  that 

(60)  11   =  1P.1  +  7Lp   (i  >  P)   » 

which  defines  a  type  of  recurring  series  [8]« 
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6»   The  Number  of  Orderln^s  whose  Error  Matrix  has  the 
Same  Eigenvalues  as  the  Usual  Orderlng« 

As  we  saw  In  Section  l^.,    each  solution  of  the   f .  . 
equations  gave  rise  in  general  to  more  than  one  possible 
ordering.   The  problem  of  counting  the  number  of  orderlngs 
equivalent  to  the  usual  ordering  for  a  general   p   seems 
very  difficult  from  the  present  point  of  view.   As  we 
have  not  been  able  to  solve  this  problem,  we  will  content 
ourselves  In  showing  why  the  problem  Is  so  difficult. 

We  first  give  a  classification  to  the  solutions  of 
the    V.  .      equations  (28-9).   By  substitution, 

(61)         V^j^H^-H. 

Is  a  solution  of  (28).   If  we  normalize  the  ^^      solutions, 
since  only  the  difference  is  relevant,  by  setting  0^  =  1, 
all  remaining  0.      can  be  determined  when  the   rt"  .   are 
given.   First  note  that  since   r^  .  =  0  or  1,   max(j2$.)  <  1. 
In  order  to  find  the  min(j^.  ),   consider  the  following 
array 

Line  1    ^^-^^     0^-f6^    ...   J2^i-^p+i 

Line  2      1    0^-0^    ...  f^^'^-p-^l     ^2~^p+2 

Line  3  1   ...  ^3-^p+2  ^3"^p+3 


Line  n-1  1      ^n-l'^n 

Line  n  1 
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This  Is  the  array  of  exponents  of  the  upper  half  of  the 
error. matrix.   Since  0,  =  1   and   f^.  .  =  0  or  1^ 

i2^2»^3»***»i^p+l  =  0  or  1. 

To  find  inln(0.),   start  on  the  first  line  of  the 
array  and  decrease  0.   consistent  with  the  requirement 
that  0^-0 A   =  0  or  1.   On  line  1  0^  =  1,   0^   =  ^^   =   <>,.= 

-  0   +-I    ~  ^'      ^^   line  2  0p   through  i^„+-i   is  determined 
by  the  choice  on  line  !•   However  we  can  choose  i^^+p  * 
and  the  smallest  value  it  can  take  on  is  j^^+p   ~   ~1»   On 
line  3  all  jZ^.'s   are  determined  by  lines  2  and  1  except 
^      o  which  must  be  chosen  as  i^  .o  ~  "1»   ^®  ^^^   forced 
to  choose  the  one  undetermined  0.      by  the  previous 
lines  as  -1  until  we  arrive  at  line  p+2«   All  fi.      on 
this  line  are  determined  except  ^p  ^p.      Since   i^„^.-|  " 

-  02.-P+2   =  0  °^  1   ^^^  since  0^+2.   ~   ~^ '      ^^®  ^^^^^^v+Z^' 
=   -2»   This  process  can  be  continued  until  all  the  rows 

are  exhausted. 

We  see  that  the  minimum  length  of  a  string  of 

constant  0.      is   p,   since  the  undetermined  0.      on  a 

line  could  have  been  set  equal  to  the  other  j^.'s. 

Define   [n/p]   as  the  largest  integer  contained  in  n/p, 

then 

(62)  minCjZf    )   =  -[§]+  i 

since    this    is   the   nijmber  of  minimum   strings   that   can  be 
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formed  from  n  considering  the  first  and  last  ^.      to 
be  a  string  of  minimtim  length.   For  example  if  n=6   and 
p=2,   the  values  of  ^.     which  produce  min(0.)   are 

h      4   ^3  K       H       ^6 
1    0    0   -1    -1   -2   o 

We  can  summarize  these  results  in  the  following 
Lemma t   A  representation  of  the  solution  of  the  7^ .  . 
equations  (28)  is 

and   if     jZl^   =  1, 

(a)  max(jZ5^)   =  1. 

(b)  j^  .      is   a   decreasing   integer  valued   function   and 

J 

(c)  min(jZJ^.)   =  1   -   [^]      . 

This  lemma   affords   us    a  classification  of   solutions 
according   to    the   number   of    strings    of   constant      0  .«      In 
the   example    of   Section  i|  where      n=5   and  p=2     we   have 

Solution   label    of   Section  \\.. 

(a) 
(d) 
(f) 

(g) 

(h) 

(b) 
(c) 
(e) 
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K 

^2 

h 

\ 

h 

1 

0 

0 

0 

0 

1 

1 

0 

0 

0 

1 

1 

1 

0 

0 

1 

1 

1 

1 

0 

1 

1 

1 

1 

1 

1 

0 

0 

-1 

-1 

1 

0 

0 

0 

-1 

1 

1 

0 

0 

-1 

The  first  group  of  solutions  (a),  (d),  (f)  and  (g)  will 
be  referred  to  as  one  jump  solutionsj  the  second  group 
of  solutions  (h)  will  be  referred  to  as  no  jump  solutions; 
the  third  group  of  solutions  (b),  (c)  and  (e)  will  be 
referred  to  as  two  jump  solutions.   The  number  of  jumps 
is  equal  to  the  number  of  constant  strings^   Since  this 
classification  seems  reasonably  natural,  we  will  try  to 
count  the  number  of  orderings  given  by  each  type  of  j^. 
solution.   Let  us  restrict  ourselves  to  the  case  of  p=2» 
Consider  one  jump  at  ^. 

^j-1   ^j    ^J+1  ^J-^2   ••• 
1     1     0     0      o.. 

and  the  corresponding  error  matrix,  where  the   A.  . 
which  are  irrelevant  in  the  present  context,  are 
suppressed. 


(63) 


\ 


j-1         0    ...    0         cr       CT        (T       1         (5"        0 

j  o*''  0      <f     6'     <r      <r     cf     0 


o       o       o 


o       o       o 


j+1        0    . .  .  0        1        1        C       1        1        0 


A.  W  »  «  «  : 

1        1        0    ..,/ 


j+2      \  OlcrcrllO..,/ 

•■.  ■■■  •■.   ■■   ■.  / 

Designate   e .   as  the   j     equation  and  let  the 
symbols   e.  <  e.  mean  equation   i   is  corrected  before 
equation   jj   then  a  one  jump  ^.       solution  implies  the 
ordering 
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(6I4.)   e^  <  Sp  <  . . .  <  e  . 

(66)  e  .^2  "^  ®n- 

(67)  e^.^^  <  ej.^2  <  '••  <  ®n 

as  Can  be  easily  seen  from  the  form  of  the  error  matrix 

th 
given  by  (63) J  e.g.  the   j    equation  cannot  be  corrected 

until  the   ,i+2    equation  is  corrected  which  is  the 

meaning  of  (66). 

The  problem  of  counting  the  number  of  equivalent 

orderlngs  has  been  re-expressedj  i.e.  to  counting  the 

number  of  arrangements  of  the  symbols   e.   subject  to 

constraints  of  the  form  (6I|.-67).   When  there  are  two 

jumps  to  the  j6.      solution  there  will  be  two  sets  of 

constraints  J 

(68 )  e,  <  ep  <  ,. .  <  e  . 

(70)  e.^^^<e.^ 

( 71 )  e  .   ,  <  e  .   5  <  ...  <  e  . 

•^1      ^1  "^  -^2 

^^^^   ^V^  ^  ^J2+2  "  •••  <  ^n   • 
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Even  though  the  solutions  can  be  typed  in  a  fairly  neat 
way,  it  seems  impossible  to  count  the  number  of  equivalent 
orderings. 

In  the  case  of  one  jump  (614.-6?)  it  is  possible  to 
count  the  niomber  of  equivalent  orderings.   Let  us 
consider  the  following  urn  problem:   How  many  ways  can   j 
white  balls  and  n-j  black  ball's  be  arranged  in  n 
•arnsV   This  problem  is  a  simple  problem  in  combinatorial 
analysis  whose  answer  is  the  coefficient   a*^p      in  the 
expansion  of   (a+6)  «   Looking  at  one  of  these  arrange- 
ments and  calling  the  first  white  ball   e,  ,   the  second 
white  ball  e^,    >..;  and  the  first  black  ball  ^'+-i» 
the  second  black  ball   e.,„,  «.»«   we  would  have  an 
arrangement  satisfying  {6l\.)    and  (67)c   Some  of  the 
arrangements  would  satisfy  (65)  and  (66)  and  some  would 
not.   If  we  could  now  enumerate  the  arrangements  that 
do  not  satisfy  (65)  and  (66)  and  subtract  them  from  the 
urn  problem  count,  we  would  have  those  arrangements  that 
would  satisfy  the  restrictions  (6l4.-7)«   Summing  this 
over   j   from  1   to  n-1,  we  would  have  the  total 
number  of  orderings  specified  by  the  one  jump  0. 
solutions.   Going  to  the  two  jump  0.      solutions 
(68-7i|),  we  could  consider  the  urn  problem  of  distri- 
buting  jn  white  balls,  2p~2-\    black  balls   and  n-jp 
orange  balls.   The  number  of  arrangements  is  given  by 
the  proper  coefficient  of  the  trinomial  expansion 
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(a+p-Hy")  •   Some  of  these  arrangements  will  violate  the 
conditions  (69),  (70),  (72)  and  (73).   Proceeding  as 
before  we  would  find  the  number  of  orderings  specified 
by  the  two  jump  0.      solutions.   Continuing  on  this  way, 
we  would  try  to  enumerate  all  the  orderings.   The 
success  of  this  approach  depends  on  the  enumeration  of 
the  exceptions  to  the  simple  urn  problems  described  above, 
The  solution  of  the  urn  problems  is  simple,  because  we 
need  only  find  the  proper  coefficient  of  an  expansion 

of  a  symmetric  function  [9]«   The  exceptions  (65),  (66) 
and  (69),  (70),  (72),  (73)  seem  not  be  be  countable  by 
a  coefficient  of  an  expansion  of  a  symmetric  function 
and  therein  lies  the  difficulty. 
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